Cyclical Local Structural Risk Minimization with Growing Neural Networks

نویسنده

  • Jan Matti Lange
چکیده

With that paper a new concept for learning from examples called Cyclical Local Structural Risk Minimization (CLSRM) minimizing a global risk by cyclical minimization of residual local risks is introduced. The idea is to increase the capacity of the learning machine cyclically only in those regions where the eeective loss is high and to do a stepwise local risk minimization, restricted to those regions. An example for the realization of the CLSRM principle is the TACOMA (TAsk Decomposition , COrrelation Measures and local Attention neurons) learning architecture. The algorithm generates a feed-forward network bottom up by cyclical insertion of cascaded hidden layers. The Output of a hidden unit is locally restricted with respect to the network input space using a new kind of activation function combining the local characteristic of radial basis functions with sigmoid functions. The insertion of such hidden units increases the capacity only locally and leads nally to a neural network with a capacity well adapted to the distribution of the training data. The performance of the algorithm is shown for classiication and function approximation benchmarks.

برای دانلود رایگان متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Boosted ARTMAP: Modifications to fuzzy ARTMAP motivated by boosting theory

In this paper, several modifications to the Fuzzy ARTMAP neural network architecture are proposed for conducting classification in complex, possibly noisy, environments. The goal of these modifications is to improve upon the generalization performance of Fuzzy ART-based neural networks, such as Fuzzy ARTMAP, in these situations. One of the major difficulties of employing Fuzzy ARTMAP on such le...

متن کامل

Dyadic Classification Trees via Structural Risk Minimization

Classification trees are one of the most popular types of classifiers, with ease of implementation and interpretation being among their attractive features. Despite the widespread use of classification trees, theoretical analysis of their performance is scarce. In this paper, we show that a new family of classification trees, called dyadic classification trees (DCTs), are near optimal (in a min...

متن کامل

Comparison Study on Neural Networks in Damage Detection of Steel Truss Bridge

This paper presents the application of three main Artificial Neural Networks (ANNs) in damage detection of steel bridges. This method has the ability to indicate damage in structural elements due to a localized change of stiffness called damage zone. The changes in structural response is used to identify the states of structural damage. To circumvent the difficulty arising from the non-linear n...

متن کامل

A New Topology for Artificial Higher Order Neural Networks: Polynomial Kernel Networks

AbstrAct Aiming to develop a systematic approach for optimizing the structure of artificial higher order neural networks (HONN) for system modeling and function approximation, a new HONN topology, namely polynomial kernel networks, is proposed in this chapter. Structurally, the polynomial kernel network can be viewed as a three-layer feedforward neural network with a special polynomial activati...

متن کامل

A Novel Extreme Learning Machine Based on Hybrid Kernel Function

Extreme learning machine is a new learning algorithm for the single hidden layer feedforward neural networks (SLFNs). ELM has been widely used in various fields and applications to overcome the slow training speed and over-fitting problems of the conventional neural network learning algorithms. ELM algorithm is based on the empirical risk minimization, without considering the structural risk an...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

عنوان ژورنال:

دوره   شماره 

صفحات  -

تاریخ انتشار 1996